In recent years, there has been a technique that estimates a position and orientation of a camera equipped in a personal computer (PC), a mobile terminal, a wearable terminal, or the like and, by using an estimation result, superimposes additional information onto an image displayed on a screen of the camera to support user operations.
During a user operation, since a camera position and orientation may frequently change, estimation of the camera position and orientation is likely to be temporarily unsuccessful. It is therefore desirable that a process of estimating a camera position and orientation be resumed from a state of unsuccessful estimation. In the following description, a process of resuming estimation of a camera position and orientation from a state of unsuccessful estimation may be referred to as a relocalization process.
An example of the related art for a relocalization process will be described. In the related art, keyframes each associating an image at the time of successful estimation with a camera position and orientation are generated intermittently at timings when a camera position and orientation are successfully estimated. In the related art, in attempting a relocalization process, the current camera position and orientation are estimated by detecting a keyframe that is close to a current image taken by the camera and associating a feature point of the detected keyframe with a feature point of the current image taken by the camera.
There is another related art that selects a set of feature points having a smaller error from a plurality of sets of feature points associating a keyframe with the current image and uses the selected set of feature points to perform a relocalization process.
Related arts are disclosed in U.S. Pat. No. 8,836,799 B2 and Japanese Laid-open Patent Publication No. 2012-221042, for example.